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ABSTRACT 

The institution in this study is a medium size, public liberal arts institution located in a rural 
setting. At the beginning of Fall 2012 term, 5,366 undergraduate and 922 graduate students were 
enrolled in the university’s ten-week quarter system. In an attempt to increase student success 
and retention, the institution implemented a student success initiative. This initiative used an 
early alert and intervention program to prompt targeted intrusive advising appointments with 
students marked at-risk by faculty or staff. A student success analysis based on student contact 
was conducted to identify any correlations between the population of students who responded 
and received an intervention, compared to the population that did not respond. Seven hundred 
and thirty two requests for intervention were submitted between the Winter of 2013 and the 
Spring of 2014 (excluding summer session); averaging 146 referrals per term. In total, 672 
students were referred using the early alert system. Of the 672, the university intervened with 
279 students (42%); 393 did not respond to intervention requests (58%), which is a consistent 
response rate when compared to other early alert research (Dobele, Gangemi, Kopanidis, & 
Thomas, 2013; Hudson, 2006). When evaluating the student population that received an 
intervention, 137 of the 279 were considered successful or 49%. One hundred and thirty one 
(47%) did not demonstrate positive academic progress and 11 (4%) were considered neutral. 

The population of students that did not respond showed an increased level of negative academic 
progression. Of the 672 referred students, 393 did not respond to a request for intervention and 
52%, or 204, did not demonstrate positive academic progress. Additionally, only 41%, or 161, 
were considered successful and 7% (28) were considered neutral. In contrast, the student 
population that received an intervention had a 49% success rate; a success rate difference of 8%. 
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Overview 

The university is a medium size, public liberal arts institution that operates on a ten-week 
quarter system. At the beginning of the Fall 2012 term, 5,366 undergraduate and 922 graduate 
students were enrolled, totaling 6,188 students. This university primarily serves first-generation 
college students and individuals coming from a low-income background. Recognizing the need 
to indentify students at-risk of failing or attrition, the institution decided to adopt an early alert 
and intervention program. A new student success specialist position, situated in the university’s 
advising office, was created and tasked with the design, development, and implementation of the 
program. This was done intentionally for one primary reason; effective academic advising is 
seen as a key element to student success and retention (Drake, 2013; Nutt, 2003; Swanson, 2006; 
Tinto, 1993) and thus, housing such a position within the university’s general advising office was 
deemed most appropriate. 

The ideology of advising has existed in a variety of forms since the outset of higher 
education in the United States (Gillispie, 2003). As academic advising evolves, professionals are 
faced with numerous circumstances. These circumstances, once a rather linear process, have 
transformed into a complex consecution of intercommunication that emulate a multitude of 
academic, developmental, personal, social, and institutional problems (Drake, 2013). The 
transformation in the field of advising has led many institutions to shift their approaches with 
regard to student services. 

As the increased complexity of student issues continue to grow (outlined below), research 
regarding the validity of advising as it relates to student success has increased (Bahr, 2008; 

Engle, Tinto, & Pell Institute for the Study of Opportunity in Higher E., 2008; Heiss-Arms, 


Cabrera, & Brower, 2008; Kuh, Kinzie, Schuh, Whitt, & Associates, 2005; Kuh, Kinzie, Schuh, 
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& Whitt, 2011a; Nutt, 2003; Schwebel, Walbum, Klyce, & Jerrolds, 2012; Swanson, 2006; 
Young-Jones, Burt, Dixon, & Hawthorne, 2013). The increase of modem research within the 
field insinuates that institutions have either started to shift towards a more student success 
oriented approach to advising, rather than simply assisting students with choosing coursework 
for the subsequent term, or research regarding student success and retention has increased. 

Advisors today are addressing issues related to finance, personal relationships, decisions 
about coursework, choosing a major and minor, and academic progress (Drake, 2013; Miller, & 
Murray, 2005), in addition to study skills, learning styles, and note taking methods. The role of 
the student success specialist encompassed all aspects of general advising as well as information 
related to student success and university processes. 

When a faculty or staff member refers a student to the student success specialist using the 
web-based system, an automated email is sent to the student notifying them that a faculty or staff 
member is concerned about their academic progress. The email provides information regarding 
tutoring, withdrawal processes, and important deadlines. The email also recommends that the 
student schedules an appointment with the student success specialist to discuss the issues further. 

Early Alert 

Electronic early alert and intervention systems provide institutions with a methodical 
process used to identify and intervene with students who are exhibiting at-risk behaviors 
(Tampke, 2013). In the early history of American higher education, student success and 
retention was not important (Siedman, 2012). Only in the last few decades of the twentieth 
century did the demand for higher education and aspirations for degree achievement increase and 
student success and retention became important (Siedman, 2012). Web-based early alert 
programs can vary in structure and application (Tampke, 2013). For example, Bowen, Price, 
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Lloyd, and Thomas (2005) assessed the effectiveness of early interventions as it related to 
attendance tracking, while other research has focused on midterm grade reporting (Cuseo, 2004). 
Some early alert software goes as far as offering predictive analytics by combining an 
institution’s enrollment information with data gathered from student surveys and subsequently 
identifies the strengths and weaknesses of each student to further assist the institution in making 
meaningful decisions related to current and future retention initiatives (Noel-Levitz, n.d.). The 
early alert system used in this study was targeted at undergraduate students and offered a wide- 
range of reasons for the referral (See methods section for more details). 

Research conducted by Hudson (2006) provides promising results for early alert systems 
as it relates to excessive absenteeism and the retainment of freshman students. Specifically, 216 
students were reported as having excessive absenteeism in the 2"‘^, and 6^’’ week of the 
semester. Of the 216 referred students, 108 (50%) received a contact attempt, however 91 of the 
108 responded to the contact attempt. Additionally, Forty-four (48%) of the 91 contacted 
students remained in the course in which they were referred from and passed, compared to the 33 
(36%) who remained in the class and failed. Fourteen of the 91 students dropped the course. In 
addition to the 48% success rate, students reported being surprised that their attendance was 
being tracked and shocked that “. . .someone cared enough to contact them and offer guidance 
and assistance.” (Hudson, 2006, p. 225). 

Chappell (2010) also assessed the use of an early alert system to report students who 
were not attending class. Faculty could electronically record if a student had missed class by 
using their electronic class roster. Email notifications were sent to students who were reported as 
absent. Similar to this study, each referral was assigned to an academic advisor. Findings 


showed an increase in student success outcomes from 52% to 66%. 
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While it appears that most early alert programs rely on increased levels of student contact 
as part of their method to foster student success, this study included, other research has indicated 
that . .simply engaging in the [intervention] process is not enough to ensure improvement.” 
(Dobele et ah, 2013, p. 69); however, Dobele and colleagues also noted that students who engage 
in the at-risk process are less likely to become at risk again, compared to those students who do 
not engage. Literature regarding student engagement and success is not sparse; however, despite 
the growing trend of early alert programs, scholarly research regarding the practices is infrequent 
(Fletcher, 2012). 

Despite the present gap in literature, findings from a national survey conducted by 
Fletcher (2012) targeted at 1,778 four- year institutions across the United States indicated 81.4% 
of respondents (529) were moderately-to-very satisfied with their early alert program. 

Implementation 

The university purchased the web-based early alert system in the summer of 2012. 

Design and development began at the start of the Fall 2012 term with a pilot program scheduled 
for release in Winter 2013. There were two implementation environments to consider: 1) the 
web-based early alert system, and 2) the integration of the system into the university 
environment. 

The early alert system component was rather systematic and included time for discovery, 
during which the company contracting the early alert system and the university’s implementation 
team established a timeline, in addition to identifying the type of student information system 
(SIS) used by the university. From there, computer programming and scripting was done to 
automatically select specific information from the SIS system and upload the data into the early 
intervention program using a Secure Shell File Transfer Protocol method, commonly referred to 
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as SFTP. Because the university housed its own student Portal, Single sign-on (SSO) protocols 
had to be created to auto authenticate users into the system. This process made accessing the 
system for faculty and staff easy by logging in the university Portal page, then simply clicking on 
the early alert icon. Additionally, administrator training was conducted in a simulated 
environment during the Fall 2012 term. University- wide training was conducted by the student 
success specialist both in person and via instructional video. 

During the Fall 2012 development of the early alert system, emails were sent to the deans 
of the College of Education and the College of Liberal Arts and Sciences asking for interested 
departments to participate in the Winter term pilot program. Ultimately, faculty from Math, 
Biology, Criminal Justice, and Health/Physical Education volunteered to participate. Student 
service offices, such as the Student Enrichment Program (TRIO) and the Multicultural Student 
Services Program also participated. In the Winter of 2013, campus collaboration efforts began in 
an effort to obtain buy-in from stakeholders (i.e., faculty and staff). Presentations were given at 
the College of Education and the College of Liberal Arts and Sciences department and division 
meetings across campus. Question and answer sessions were held and demonstrations were 
provided to faculty and staff. At the beginning of the Spring 2013 term, the program went live 
campus-wide. 


Method 

Definitions 

The university had a modified three-strike academic standing policy. That is, if a student 
had three consecutive terms with a term GPA less than 2.0 the student would be suspended. It is 


important to understand how the university standing policy is constructed as academic standing 
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was used to assess student success in this study. Below is a chart that further exemplifies the 


process. 


Figure 1 



For the purpose of this report as it relates to student success, an intervention is considered 
successful if any of the following conditions are true: 


1. A student is referred AND: 

a. Was contacted by advising staff regarding the RFI and 

b. The student is in good standing and stayed in good standing at the conclusion of the term. 

2. A student is referred AND: 

a. Was contacted by advising staff regarding the RFI and 

b. The student was, at the time of the referral, in academic warning status but returned to good 
standing at the conclusion of the term. 

3. A student is referred AND: 

a. Was contacted by advising staff regarding the RFI and 

b. The student was, at the time of the referral, in academic probation status but returned to good 
standing at the conclusion of the term. 

4. A student is referred AND: 

a. Was contacted by advising staff regarding the RFI and 

b. The student was, at the time of the referral, on academic warning status AND continued on 
academic warning (continued academic probation) at the conclusion of the term. 


Some students in this report are considered neutral. This occurs when: 
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1 . A student was in bad academic standing and maintained that same standing at the conclusion 
of the term. For example, this may occur if a student was on academic warning and then 
withdrew from the current term. Thus, the student maintains the last term attended academic 
standing. 

2. A student is a new freshman and did not have academic standing prior to the term. If the 
student then withdrew from the term, or otherwise received non-punitive grades resulting in no 
academic standing, the student would be considered neutral. 

3. A student had no immediate prior term standing due to taking the prior term off and received 
non-punitive grades in the term in which they were referred. 

4. A student had non-punitive grades in the prior term and received non-punitive grades in the 
term in which they were referred. 

Participants 

The university had a Fall 2013 undergraduate enrollment size of 5,266. Eight hundred 
and fifty eight students were new incoming freshman. Of the 5,266 students 4,423 were full- 
time and 843 were part-time (Oregon University System Fact book, 2013). Graduate students 
were not part of the study and were not listed within the early alert system. 


Procedure 

When a faculty or staff member refers a student to the student success specialist using the 
web-based system, the individual is offered a preset list of reasons for the referral. There are 
three radial button options for each question 1) “Yes,” 2) “No,” and 3) “Don’t Know.” Each 
question response is defaulted to “Don’t Know.” This allows the faculty or staff to answer only 
questions that apply while leaving the other questions as “Don’t Know.” These reasons are 
provided below. A “Comments” box was also provided at the bottom of the referral page so 
faculty and staff could add additional information related to the referral. 


1 . abrupt change in academic engagement and/or overall demeanor 

2. Excessively tardy in assignment and/or attendance 

3. Not attending class 
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a. More than two absences? 

b. Are they consecutive? 

4. Not actively participating in discussion/class 

5. Not turning in/not completing assignments regularly 

6. Failing assignments 

a. More than two failures? 

b. Are they consecutive? 

7. In danger of failing school/multiple classes 


After a referral was generated, an automated email was sent to the student notifying them 
of the concern. The email provided information regarding tutoring and withdrawal processes and 
important deadlines. The email also recommended that the student schedule an appointment 
with the student success specialist to discuss the issues further. After receiving the digital 
referral, known as a request for intervention or RFI, the student success specialist attempted to 
contact the student via telephone if the student had not scheduled an appointment or responded 
via email. If the student success specialist contacted the student via telephone, the situation was 
briefly explained and an in-person meeting was requested. While the in-person meetings were 
strongly recommended, meeting with the student success specialist was not mandatory and had 
no institutional repercussion on the student. 

Typically three attempts were made to contact the referred student, which is consistent 
with the national standard at four- year institutions based on a study conducted by Fletcher 
(2012). This included the automated email sent to the student when the referral was submitted, 
one telephone call, and an additional follow up email. If the student did not schedule an 
appointment or respond, a digital note was attached to the referral and then archived. If the 
student scheduled an appointment, the student success specialist met with the student and 


provided support and guidance in the areas of concern. 
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Analysis 

Because a large portion of early alert research is anecdotal (Fletcher, 2012), A 
comparison of the student’s previous academic standing and the academic standing for the term 
in which they were referred was conducted. Ultimately removing personal accounts of the 
interventions and basing the analysis strictly on academic performance based on GPA and 
academic standing. At the conclusion of each term, academic standing for every student who 
was referred using the early alert system was assessed. For example, if the student was referred 
during the Winter term of 2014, at the conclusion of the Winter term, academic standing for that 
student was assessed for both Fall 2013 and Winter 2014. Registration for the subsequent term 
of the referral was also recorded, in addition to whether the student responded to the request for 
intervention by meeting with the student success specialist or academic advisor within the 
advising office. 

For the purpose of this analysis, two populations were evaluated, 1) students who 
received an intervention by communicating with the student success specialist or another advisor 
regarding their referral, and 2) students who did not respond to an intervention request. After 
comparing each student’s standing, the students were placed into one of three categories 1) 
successful, 2) not successful, and 3) neutral. Because multiple referrals could be submitted 
regarding the same student, duplicates within the system were identified and removed. During 
the Summer session, the early alert program was not used and thus, no data is provided for those 
terms. 


Results 

After analyzing the data, 732 requests for intervention were submitted between the 


Winter of 2013 and the Spring of 2014; averaging 146 referrals per term. In total, 672 students 
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were referred using the early alert system (see table 1). Of the 672, the advising office 
intervened with 279 students (42%); 393 did not respond to intervention requests (58%). The 
42% response rate is consistent with other early alert research (Hudson, 2006). When evaluating 
the student population that received an intervention, 137 of the 279 were considered successful 
or 49%. One hundred and thirty one (47%) did not demonstrate positive academic progress and 
11 (4%) were considered neutral (see table 1) 

RFIs at a Glance: Table 1 


Winter Spring Fall Winter Spring Totals 

2013 2013 2013 2014 2014 


Total# of RFIs 

84 

176 

243 

132 

97 

732 

# of students 
referred 

73 

158 

226 

122 

93 

672 

#interventions 

47 

52 

102 

49 

29 

279 

# of students 
who did not 
respond 

26 

106 

124 

73 

64 

393 

# of successful 
students 

41 

66 

107 

51 

33 

298 

# of non- 
successful 
students 

21 

89 

106 

67 

52 

335 

# of neutral 
students 

11 

3 

13 

4 

8 

39 


The population of students who did not respond showed an increased level of negative 
academic progression. Of the 672 referred students, 393 did not respond to a request for 
intervention and 52%, or 204, did not demonstrate positive academic progress. Additionally, 
only 41%, or 161, were considered successful and 7% (28) were considered neutral. In contrast, 
the student population that received an intervention had a 49% success rate; a difference of 8%. 

When assessed term-by-term, beginning with the pilot in Winter 2013, analysis indicated 


a consistent increase in student success within the population of students who received an 
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intervention, compared to the population that did not respond (see table 2). Specifically, the 
population of students who received an intervention had a 10% higher success rate in the Winter 
2013, compared to the population of students who did not respond. In the Spring of 2013 there 
was a slight 3% increase; Fall of 2013 showed a more promising number of 7%, Winter 2014 
echoed with a 9% increase, and Spring 2014 indicated a 4% increase (see table 2). 


Comparing Success Rates by Population and Term: Table 2 



Winter 2013 

Spring 2013 

Fall 2013 

Winter 2014 

Spring 2014 

(Met) # of students 
successful 

28 (60%) 

23 (44%) 

52 (51%) 

23 (47%) 

11 (38%) 

(Not Met) # of 
students successful 

13 (50%) 

43 (41%) 

55 (44%) 

28 (38%) 

22 (34%) 

Difference 

(10%) 

(3%) 

(7%) 

(9%) 

(4%) 


When comparing the failure rates by population and term, there were some instances 
where the met-with population had a higher failure percentage than the not-met with population. 
Perhaps the most notable difference occurred in the pilot term of Winter 2013. Findings show a 
15% higher rate of negative academic progress in the population that received an intervention, 
compared to the population that did not, 16 (34%), and 5 (19%) respectively; however, it is 
important to note that percentage is based on the number of students who received an 
intervention and the population that did not receive an intervention and these population numbers 
are not the same (see table 3). 


Comparing Failure Rates by Population and Term: Table 3 



Winter 2013 

Spring 2013 

Fall 2013 

Winter 2014 

Spring 2014 

(Met) # of students not 
successful 

16 (34%) 

28 (54%) 

49 (48%) 

24 (49%) 

14 (48%) 

(Not Met) # of students 
not successful 

5 (19%) 

61 (58%) 

57 (46%) 

43 (59%) 

38 (59%) 

Difference 

(15%) 

(4%) 

(2%) 

(10%) 

(11%) 
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A notable difference with regard to neutral students was also discovered. Thirty nine of 
the 672 referred students were considered neutral, which makes up 6% of the total referred 
population. When comparing the population of students who received an intervention to the 
population that did not, there were twice as many neutral students in the population that did not 
receive an intervention (28 or 4%) compared to those that received an intervention (1 1 or 2%). 
While prior research has shown correlations with regard to student engagement and persistence 
(Conner et al., 2013; Engle & Tinto, 2008; Moreira, Dias, F. Vaz, & J. Vaz, 2013; Boretz, 2012; 
Community College Survey of Student Engagement, 2007; Swanson, 2006), persistence itself 
was not assessed in this study; however, the increase in neutral students who received non- 
punitive grades, including “W” grades for withdrawing, within the population that did not 
receive an intervention may indicate that increased student contact in this instance resulted in 
increased persistence and thus, increased student success in the population that received an 
intervention. 


Discussion 

Eimitations to this study are apparent, however, this is not uncommon among student 
success research as the complexities of the elements that shape such success are widespread. 
While the research focused on student success and potential financial gain, a detailed retention 
analysis was not conducted. Specifically, the reason why a student did not return was not 
investigated, which included determining if the student had graduated or not. Additionally, no 
satisfaction or feedback survey was offered to any of the students who received an intervention. 
Because of limited staffing and resources, in addition to the increased complexity of involving 
human subjects within research, the analysis was limited to archival data only, i.e., academic 
standing, interventions, tuition and fee amounts, and registration. 
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The timing in which a request for intervention was submitted and the term length, were 
notable elements that impacted this study. Instructors, on occasion, referred a student during the 
eighth or ninth week of a ten-week term. Often times the student was too far behind in their 
classes to recover. Promoting a proactive approach to faculty with regard to at-risk referrals is 
essential to the program’s effectiveness. Universities that operate on semesters will likely have 
more time to assist a referred student; however, timing of a referral is essential for all institutions 
utilizing an early alert system. Lastly, a control group was absent from the analysis (students 
enrolled in the same courses who could have been referred). Despite the missing control group, 
research conducted at Columbia College and California Sate University, Stanislaus, found that 
early alert system referrals do identify students most at-risk (Pfleging, 2002). 

Findings 

A financial analysis revealed a significant monetary benefit to the university. During the 
five terms, the early alert program supported 279 interventions in which the faculty or staff was 
concerned about the referred student’s overall success. A financial analysis was conducting by 
assessing the tuition and fee costs of the subsequent terms for the 279 students who received an 
intervention, regardless of if they progressed to good academic standing subsequent to the term 
in which they were referred. The notion behind the method is that when a student was referred, 
the student was at-risk of failing and attrition. Any registration subsequent of the intervention 
could be seen as gained tuition and fees, assuming the student was in danger of attrition. As a 
result, the early alert system aided in the future tuition of approximately $1.2 million. With 
notable financial implications such as these, it is no surprise the university doubled the yearly 
budget for its alert software and upgraded to more all inclusive student success system. 


Next Steps 
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While the data does not validate causation with regard to the relationship of early 
interventions and future registration of intervened students, the consistent increase in student 
success across all five terms within the population that received an intervention points to a 
certain level of correlation. Specifically, the results indicated an increase in student success 
within the population that received an intervention across all five terms, 60%, 44%, 51%, 47%, 
and 38%, compared to the population that did not receive an intervention, 50%, 41%, 44%, 38%, 
and 34%, respectively. Consequently, the findings support the notion that targeted student 
contact in academic advising promotes student success for at-risk students. Despite the growing 
research in the area of engagement, persistence, and student success, more research is needed to 
further identify the true extent of the relationships and provide a guide for best practices. 
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